Censoring technique and numerical computations of invariant distribution for continuous-time Markov chains

Continuous Time Markov Chains

Discrete-time Markov chains are useful in simulation, since updating algorithms are easier to construct in discrete steps. They can also be useful as crude models of physical, biological, and social processes. However, in the physical and biological worlds time runs continuously, and so discrete-time mathematical models are not always appropriate. This is especially true in population biology –...

Continuous-time Markov Chains

Lecture-25: Markov Chains: Invariant Distribution

Theorem 1.1. An irreducible Markov chain with transition probability matrix P is positive recurrent iff there exists a unique invariant probability measure π on state space S that satisfies global balance equation π = πP and πi = 1 μii > 0 for all i ∈ S. Proof. Let X be a positive recurrent Markov chain on state space S, with X0 = i. Let Hi be the first recurrence time to state i, and let N j(n...

Continuous-time homogeneous Markov chains

Verifying Continuous Time Markov Chains

We present a logical formalism for expressing properties of continuous time Markov chains. The semantics for such properties arise as a natural extension of previous work on discrete time Markov chains to continuous time. The major result is that the veriication problem is decidable; this is shown using results in algebraic and transcendental number theory.